PL/I Importer
=============

Overview
--------
Given a PL/I source file, this component produces an EMF model that represents
the data structures declared within it. Basic program flow is:

1. Invoke the IBM PL/I for Windows compiler for the source file, specifying the
   NC (no compile) and the XINFO(XMI) compiler options. The compiler is in
   com.ibm.ccl.pli.win32/importer as pli.exe.
2. The compiler performs syntax checking, and generates an XMI file describing
   the data structures in the program.
3. Load the EMF model from the XMI file.
4. Perform post-processing on the EMF model (this allows us to handle e.g.
   differences between the Windows and z/OS compilers)
5. Return the model to the caller.

There is additional logic for managing preferences, which are used to determine
compiler options. This is just standard Eclipse code.


The Model
---------
The entire logic revolves around a model of the PL/I data structures. This
originated as a Rational Rose model, PLITDLang, which is converted to an EMF
model. 

The model is made up of three subunits, pli, TDLang, and TypeDescriptor - the
idea was that TDLang and TypeDescriptor should be language independent and used
within models for multiple languages (it's also used for C), with just the
language-specific parts in a separate subunit (pli in this case). Note that
TDLang and TypeDescriptor are just copies (and out of date ones at that) - the
master copies are in com.ibm.etools.tdlang in the J2C Lang Core component.

Note that the tags in the XMI file generated by the compiler map exactly onto
the classes defined in the model - this reduces the loading of the model to
a simple EMF call. It also means that the compiler and this code need to be
kept in step. When making changes to this model, it is likely that you will
need to request a change to the compiler.

Now, it would probably make more sense to just work with the EMF model, but
that would take someone with the courage to formally make that decision. Until
then, the Rose and EMF models are kept in step.


Rational Rose
-------------
Obtain Rational Rose from the eXtreme Leverage site. Once installed, launch
Rational License Key Administrator and point it at license servers
9.20.137.112:27000 and/or 9.20.137.113:27000.

Rose appears to be a complex and unwieldy tool. Fortunately, we only need to
use the very basics to maintain the model, so it's not too bad. It does have
issues - in particular, it seems to have trouble with line breaks when editing
documentation. When this happens, it's sometimes easier to edit the raw file in
a text editor, then load the model into Rose to verify that nothing's broken.

When Rose opens, close the New Model dialog that comes up by default, and
open PLITDLang.mdl directly from the workspace. Accept the loading of subunits
when prompted and ignore the warning about a non System Default Character Set.
You can also ignore the Class Diagram view that opens - close it if you wish,
but it doesn't do any harm leaving it open.

You are solely interested in the Logical View>PLI section of the navigation
tree. Here, you can see all the enumerations, classes, and their attributes.
Add a new class by right clicking on the PLI folder and select New>Class. Add a
new attribute or enumeration value by right clicking on the class or
enumeration and select New>Attribute.

Once created, you will need to edit the new entity by opening its
Specification, modifying the attributes (compare with other existing entities
to work out what's needed) and clicking OK. Repeat with the entity's Standard
Specification. Attributes that you'll likely need to change are inheritance
and visibility (Specification), and setting all the Model Properties to their
default values (Standard Specification). The first few times, I'd suggest
doing a diff on the raw file and comparing your new class/enumeration/attribute
with an existing one until you're happy that you've got it right.

When you're done, save the file. Undo any changes to TDLang.cat and
TypeDescriptor.cat - you shouldn't be changing these.


Generating the EMF model
------------------------
The next stage of the process is to convert the Rose model to an EMF generator
model. 

First, delete the contents of the src/model folder (the tooling doesn't handle
replacing existing files). 

Then, right click on PLITDLang.mdl and select New>Other... and, in the wizard,
select Eclipse Modelling Framework>EMF Generator Model and click Next. Select
the com.ibm.ccl.pli/src/model folder, specify a file name of My.genmodel, and
click Next. Select "Rose class model" and click Next. Click Load. Ignore the
problems with the Rose model (or fix them...). Once the load is complete, click
Next. Select all of the root packages, rename typedescriptor.ecore to
TypeDescriptor.ecore, and click Finish. The model will now be created. Undo any
changes to globaldatatypes.ecore, tdlang.ecore, and TypeDescriptor.ecore (in
hindsight, it would probably work if you didn't delete these files, and then
only select pli.ecore as a root package, but I haven't actually tried this).

Note that globaldatatypes.ecore is an empty model - I suspect that it was a
default that got left behind when the Rose model was first created, and could
probably be tidied up at some point.

Now for the messy bit - the reason why it would probably be best to only work
with the EMF model: You need to edit My.genmodel and pli.ecore down to the
minimum set of changes. In particular, you must:

- revert to the original <genmodel:GenModel> tag, so that you retain the
  copyright statement that will be copied into every source file generated
  from the EMF model. Change the date if necessary.
- remove all instances of 'typeSafeEnumCompatible="false"' wherever they
  occur. Since this model was first created, the EMF implementation of
  enumerations changed in a completely incompatible way - notably, constants
  that we would expect callers to reference either don't exist any more, or
  have a different data type. The attribute as specified tells EMF to generate
  new-style code. Removing the attribute (or changing its value to "true")
  tells it to generate the old-style code - which is what we need. If you
  don't do this, then callers WILL BREAK.
  
  
Generating EMF classes and interfaces
-------------------------------------
The final stage is to generate the class and interface source files from the
model. You will probably want to disable automatic building of the project
until this stage is complete.

Open My.genmodel with the EMF Generator. Right click on the Pli subunit and
select Generate Model Code. EMF now regenerates the entire set of source files,
interfaces and enumerations into the com.ibm.ccl.pli package, classes into
com.ibm.ccl.pli.impl.

Most of the files will have no changes other than the date in the copyright
statement - you should Undo these. You will typically need to keep:

- any enumerations that you have modified in the model
- any interfaces that you have modified in the model and their corresponding
  *Impl classes 
- PliFactory and PliFactoryImpl
- PliPackage and PliPackageImpl
- PliAdapterFactory
- PliSwitch

You can Undo the rest (including plugin.properties, plugin.xml, and
MANIFEST.MF). Depending on what you have changed, you might also be able to
Undo the Factory, Package, AdapterFactory, and Switch changes - check them to
see if anything other than the copyright date has changed. Don't worry about
undoing too much - if you get it wrong, you can always start this section again
and go around as many times as you like until you get it right.

Note that only the String declaration of the copyright statement is updated
with the date from My.genmodel. You will need to update the date in the 
comment block at the top of each source file manually.

Re-enable automatic building if necessary and let the project build. If you
have build errors, this usually means that you've undone too much - go around
again.

Now test it!
